Cache-Based Matrix Technology for Efficient Write and Recovery in Erasure Coding Distributed File Systems

With the development of various information and communication technologies, amount big data has increased, distributed file systems have emerged to store them stably. The replication technique divides original into blocks writes on multiple servers for redundancy fault tolerance. However, there is a symmetrical space efficiency problem that arises from need larger than data. When storing data, Erasure Coding (EC) generates parity through encoding calculations separately each server tolerance recovery purposes. Even if specific fails, can still be recovered decoding using stored remaining servers. matrices generated during are redundantly writing recovery, which leads unnecessary overhead in systems. This paper proposes cache-based matrix uploads cache memory reuses them, rather generating new time or occurs. design applies Weighting Size Cost Replacement Policy (WSCRP) algorithm efficiently upload reuse parameters known as weights costs. Furthermore, table managed because weight–cost model sorts updates parameters, reduces replacement cost. experiment utilized Hadoop Distributed File System (HDFS) system, EC volume was composed Reed–Solomon code with (6, 3). As result experiment, it possible reduce write, read, times associated decoding. In particular, up three node failures, WSCRP were able by about 30 s compared regular HDFS